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The aim of this paper is to address the question: Can an artificial neural network 
(ANN) model be used as a possible characterization of the power of the human 
mind? We will discuss what might be the relationship between such a model and its 
natural counterpart. A possible characterization of the different power capabilities 
of the mind is suggested in terms of the information contained (in its computational 
complexity) or achievable by it. Such characterization takes advantage of recent 
results based on natural neural networks (NNN) and the computational power 
of arbitrary artificial neural networks (ANN). The possible acceptance of neural 
networks as the model of the human mind's operation makes the aforementioned 
quite relevant. 



1. Introduction 

Much interest has been focused on the comparison between the brain and 
computers. A variety of obvious analogies exist. Based on several thoughts, 
some authors from very diverse research areas (Philosophy Physics, Com- 
puter Science, etc.) claim that the human mind could be more powerful 
than Turing Machines 47 ' 52 ' 13,59 . Nevertheless, they do not agree on what 
these "super- Turing" capabilities mean. Consequently there is no univer- 
sally accepted characterization of this extra power and how it could be 
related to the human mind, even though there is strong defense of these 
authors' theories based on whether or not humans are "super-minds" ca- 
pable of processing information not Turing-computable. 
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Nowadays, it is accepted that the nervous system, including the spinal 
cord and the neocortex, control behavior. In a simplified picture, the ner- 
vous system could be viewed as a device which receives input from various 
senses and produces output in the form of some action. From a computa- 
tional point of view, the word "produces" could be harmlessly substituted 
by "computes" , in the sense that the mind computes in the way a Turing 
machine does. One of the most interesting and challenging tasks in sci- 
ence is to assume this as a possibility in order to understand how these 
computations are performed. Thus, it is not surprising that the field of 
Neuroscience attracts many researchers not only from the fields of Biology 
and Computer Science, but also from the area of Physics. 

Contributions to the mind's neural computation require a strong effort 
towards interdisciplinary collaboration. As Teuscher and Sipper 65 pointed 
out, there are very few problems stemming from neural computation on 
which a theoretical computer scientist can commence work without further 
interaction or collaboration with neuroscientists. 

Since the artificial model was inspired in the natural, in this paper we 
will explore the bridge between them on the basis of recent work. Since arti- 
ficial neural networks have been inspired from their biological counterparts, 
it is natural to ask about the feedback from the artificial to the biological. 
We attempt build such a bridge by taking the standard model of ANNs and 
the way in which ARNN admits arbitrary weights, in order to character- 
ize its computational power by using the complexity of the weights or the 
timing of the pulses. 

Most of what happens in our brain does so without our being aware of 
it, so that many things that seem easy to us involve highly complex neu- 
ral circuits and operations. Most practical applications of artificial neural 
networks are based on a computational model involving the propagation of 
continuous variables from one processing unit to the next. 

Inspired by a simplistic vision of how messages are transferred between 
neurons, computer scientists invented the artificial computational approach 
to target a wide range of problems in many application areas. Biological 
neurons are connected by synapses, which are the links that carry messages 
between them. Using synapses neurons can carry pulses to activate each 
other with different threshold values. Neurons are the computational part 
of the network while links connect different neurons and enable messages to 
flow among them. Each link is a connection with a weight that affects the 
carried message in a certain way. In fact each link receives a value from an 
input neuron, multiplies it by a given weight, and then passes it to another 
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neuron. Between this process several sources of computational power can 
be identified. This means that the source of the computational power of 
an NN might have at least three possible sources: (a) one carried by the 
operations in the neuron itself; (b) the message between neurons involving 
pulses, action potentials (APs) and timing and (c) the complexity of the 
neural network weights in terms of their capability to encode information 
which implies (a) and (b). 

As biologically-inspired devices Artificial Neural Networks (ANNs) have 
been suggested as a computational theory of mental activity. An ANN is 
mathematically defined as a weighted directed graph in which each vertex 
represent a neuron and each arrow a connection. 

Definition 1. An Analog Recurrent Neural Network or ARNN 11 is 
a finite network of nodes (called neurons) and connections, wherein the 
synaptic weight associated with each connection (determining the coupling 
strength of the two end-nodes) is a real (analog) value. 

An ARNN is basically an artificial neural network allowing loops (re- 
current) and irrational (even non- Turing computable) numbers as weights 
(analog). 

We consider the question about the possible computational power of 
the human mind in this paper as a problem of computability in general and 
tractability or practical computability out of its scope then making only 
emphasis on the computability power defined by the Turing degrees and 
beyond. 

The key arguments which form the bases of our position are: 

(1) ARNNs are a plausible model of the human brain because they 
define a wide range of different levels of computational power. 

(2) At least some characteristics of the human mind arise from the 
human brain. 

(3) ARNNs are therefore a plausible model of the human mind. 

(4) It has been argued that the human mind could be depicted or 



a T/i.e ARNN dynamics is given by a mapping: 



in which each neuron Xi in time (t+1) behaves as described by the following non-linear 
eguation: 



F : R N X{0, 1} 



M 
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fully simulated by Turing machines, and others, as it has a "super- 
Turing" computational power. 

Therefore, prima facie, it seems reasonable to explore claims on 
the human mind's supposed Turing or "super- Turing" capacities 
through an analysis of ARNNs in relation to recent work on the 
brain's own neural network. 

That an analysis of ARNNs supposed Turing and "super- Turing" 
capabilities (and other more well established findings) gives rise to 
a taxonomy of computational capabilities. 

The taxonomy of computational capabilities gives rise to prima facie 
reasonable hypotheses concerning the human mind. The contribu- 
tion of this paper to our knowledge is to build up this argument and 
generate hypotheses binding artificial models to the natural. 

With reference to argument number 1, the ARNN model even allows to 
consider a situation in which the weights and the ARNN become equiva- 
lent in power to automaton of lower power (including the Turing machine). 
However, the ARNN (and a wider generalization proposed in this paper, 
to be fully explored in another, yet to be published) allows us to consider 
simultaneously, all possible scenarios. Claim number 2 is widely accepted 
in different degrees of engagement. It is the case of the mind/body problem 
and our claim is based on the consideration of the mind as the brain op- 
eration. Hence, no distinction is made between brain and mind. However, 
it is our assumption that the mind does not subtract any computational 
power and fully inherits the whole computational power of the brain. If 
mind adds some computational power to the overall system our arguments 
still apply as lower bound. By contrast, if the mind subtract computational 
power, our claims should be reformulated in terms of it and our arguments 
will lay as upper bound. 

Our intent is to discuss the mathematical features which a model should 
possess, if it is to aspire to certain computable power. Our choice of an 
ARNN model rests on two considerations: (a) it has been proposed as a 
model with (potentially) non computable capabilities (provided some re- 
quirements are met; see later); (b) as a neural network model, neurosci- 
cntists might find it easier to relate it to their own empirical work -most 
of their research is based on (pseudo)analog values- and therefore they can 
put forward and test concrete hypotheses to confirm or refute the model. 
Additionally, the ARNN represents a refinement on what had been typically 
regarded as the mathematical definition of an ANN, which did not make 
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any distinction between the complexity of the weights in terms of the com- 
putational power beyond the seminal work of Minsky. Nevertheless, neural 
networks which run on digital computers operate on a Turing-computable 
subset of irrational numbers, a strong restriction that determines a pri- 
ori its computational power. Hence, some enthusiasm generated by im- 
portant experimental and theoretical results cannot be extended easily to 
applications because there is no straightforward way to make real numbers 
available even assuming its possible existence. Digital hardware implemen- 
tation uses a finite number of bits for weight storage and rational restraint 
values for firings rates, weights and operations and remain limited to a 
computational power. Even analog implementations, which are often cited 
for their ability to implement real numbers easily (such as analog quanti- 
ties), are limited in their precision by issues such as dynamic range, noise, 
VLSI b area and power dissipation problems. Thus, in theory and in prac- 
tice, most implementations use rational numbers, or at most, a subset of 
irrational numbers, the Turing-computable ones. 

The classical approach of computability theory is to consider models op- 
erating on finite strings of symbols from a finite alphabet. Such strings may 
represent various discrete objects such as integers or algebraic expressions, 
but cannot represent general real or complex numbers, even though most 
mathematical models are based on real numbers. The Turing machine 67 and 
its generalization in the form of the Universal Turing machine (UTM) is the 
accepted model for computation, while under the Church- Turing thesis, it 
is considered the authoritative model for effective computation. However, 
some researchers have proposed other models in which real numbers play 
the main role in effective computations. 

Machines with "super- Turing" capabilities were first introduced by Alan 
Turing 69 , which investigated mathematical systems in which an oracle 
was available to compute a characteristic function of a (possibly) non- 
computable set. The idea of oracles was to set up a scheme for investigating 
relative computation. Oracles are Turing machines with an additional tape, 
called the oracle tape, which contains the answer to some non-computable 
characteristic functions. An oracle machine is an abstract machine used 
to study decision problems. It can be visualized as a Turing machine with 
a black box, called an oracle, which is able to determine certain decision 



b Very Large Scale Integration are systems of transistor-based circuits into integrated 
circuits on a single chip. For example, the microprocessor of a computer is a VLSI 
device. 
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problems in a single step. However, Turing gave no indication on how such 
an oracle might be implemented. The Turing machine can write an input 
on its own tape, then tell the oracle to execute it. In a single step, the 
oracle computes its function, erases its input, and writes its output to the 
tape. It is possible to posit the existence of an oracle, which computes 
a non-computable function, such as the answer to the halting problem or 
some equivalent. Interestingly, the halting problem still applies to such 
machines; that is, although they can determine whether particular Turing 
machines will halt on particular inputs, they cannot determine whether ma- 
chines with equivalent halting oracles will themselves halt. This fact creates 
a hierarchy of machines according to their Turing degree, each one with a 
more powerful halting oracle and an even more difficult halting problem. 

With such a method, an infinite hierarchy of computational power can 
easily be constructed by positing the existence of oracles that perform pro- 
gressively more complex computations which cannot be performed by ma- 
chines that incorporate oracles of lower power. Since a conventional Turing 
machine cannot solve the halting problem, a Turing machine with a Halt- 
ing Problem Oracle is evidently more powerful than a conventional Turing 
machine because the oracle can answer the halting question. It is straight- 
forward to define an unsolvable halting problem for the augmented machine 
with the same method applied to simpler halting problems that lead to the 
definition of a more capable oracle to solve that problem. This construction 
can be continued indefinitely, yielding an infinite set of conceptual machines 
that are progressively more powerful than a Turing machine. This build 
the hierarchy of Turing degrees: 

Definition 2. The Turing degree of a subset A of natural numbers is 
the equivalence class consisting of all subsets of B equivalent to A under 
Turing reducibility . The degree of a set A is denoted by deg(A). The least 
element in the partial order is denoted by and is the degree of all recursive 
sets (computable sets). 

In other words, two sets of natural numbers have the same Turing degree 
when the question of whether a natural number belongs to one can be 
decided by a Turing machine having an oracle that can answer the question 
of whether a number belongs to the other, and vice versa. So the Turing 
degree measures precisely the computability or incomputability degree of a 
subset X. 

Definition 3. A Turing reduction from a A to B is a reduction which 
easily computes A assuming B, i.e. A is computable by a Turing machine 
with an oracle for B. 
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Because a language can be always codified by a subset of natural num- 
bers, in terms of computability theory, sets and languages are equivalent. 

Definition 4- Formally, a language L is recognized by a Turing ma- 
chine with an oracle A if for every string s the machine using A as an oracle 
halts on input s if sGL. A language B is Turing-reducible to a language A 
if a Turing machine equipped with an oracle for A can decide B. 

Models of hyper-computation tend to be of two general types: One uses 
oracles or oracles in disguise, and the other uses infinite computation in fi- 
nite time. Whether or not such machines arc physically constructible -most 
experts believe they are not- studying them improves our understanding of 
the recursion theory. 

On the other hand neural networks have been studied as computational 
devices. In 1956, Kleene showed 27 how to simulate finite automata using 
McCulloch and Pitts neurons 41 and proved that when the weights of the 
networks are constrained to be integers, the languages accepted by them are 
exactly the regular languages. On the other hand, Minsky showed that 
neural networks with boolean neurons can simulate any finite automaton. 
More recently, Hava Siegelmann 59 presented a computation model based on 
analog recurrent neural networks (ARNN). All this work establish a very 
strong connection between artificial neural networks and automata theory. 
Both automaton (including the Turing machine) and neural networks are 
characterized by the languages they accept and hold the same hierarchy. 

Siegelmann offered a proof 59 in which the set of languages accepted by 
networks with rational weights are exactly the recursively enumerable lan- 
guages . Siegelmann showed that ARNNs are strictly more powerful than 
the Turing machine model in that they can perform computations provably 
uncomputable by a universal Turing machine. Disregarding the fact that 
it seems unrealistic for most experts that those ARNN could be someday 
constructible it is not easy to discard if such devices are already present in 
nature taking advantage on the physical properties of the universe or, at 
least, the way in which they can perform computations over its physical en- 
vironment d . According to G. Kreisel himself it is an open question whether 



c Davis 19 rightly pointed out that even if a subset of non rational numbers is used, 
namely the set of Turing-computable irrationals, the class of languages recognized by 
neural networks remains the same, as Sicgclmann's proof on the power of networks with 
rational weights readily extends to nets with computable irrational weights (as Turing 
already did with his machines). 

d Another interesting question raises by its own right: if there exists a natural device 
with such capabilities how might we be restricted to take advantage of the same physical 
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there is any "natural phenomenon" leading to an uncomputable number 9 . 

These neural networks satisfy some classical constraints of computa- 
tional theory: the input, output, and transitions, are discrete and finite. 
But the overall system is not really finite since it uses arbitrary real num- 
bers, and it is known that it is powerful enough to encode all non-recursively 
enumerable languages. Consequently, a connection between the complexity 
of the networks in terms of their information complexity and their compu- 
tational power had been developed, spanning a hierarchy of computation 
from the Turing model to a "super- Turing" mathematical model. 

2. Modeling the Brain 

A clear difference between the brain and the computer is that a digital 
computer operates by performing sequential instructions from an input 
program, whereas there is no clear analogy of such a performance in the 
human brain. However, since any Turing machine working on several tapes 
is finally equivalent to a single tape Turing machine, the difference may 
be functional rather than fundamental. The brain as a fully parallel de- 
vice, could be reduced to a sequential input under some very significant 
restrictions (for example being discrete or finite). Nowadays, such ques- 
tions are the subject of scientific and philosophical debate since we have 
the computational resources to develop live experiments 6 . However, Digital 



properties in order to build an artificial equivalent device? Much of the defense of the 
work mentioned above have precisely centered on questions such as if we are taking 
advantage of the resources we have in nature. 

c There are at least three important projects currently running: A Linux cluster running 
the MPI NeoCortical Simulator (NCS), capable of simulating networks of thousands 
of spiking neurons and many millions of synapses, was launched by Phil Goodman at 
the University of Nevada. Blue Brain, a 4,000 processor IBM BlucGene cluster, was 
used to simulate a brain in a project started in May 2005 in the Laboratory of Neural 
Microcircuitry of the Brain Mind Institute at the EPFL in Lausanne, Switzerland, in 
collaboration with lab director Henry Markram. It has as its initial goal, the simulation 
of groups of neocortical columns which can be considered the smallest functional units of 
the neocortex. Also running is the NCS, to be combined with Michael Hincs' NEURON 
software. The simulation will not consist of a mere artificial neural network, but will 
involve much more biologically realistic models of neurons. Additionally, CCortex, a 
project developed by a private company Artificial Development, planned to be a complete 
20-billion neuron simulation of the Human Cortex and peripheral systems, on a cluster 
of 500 computers: the largest neural network created to date. Different versions of the 
simulation have been running since June 2003. CCortex aims to mimic the structure of 
the human brain with a layered distribution of neural nets and detailed interconnections, 
and is planned to closely emulate specialized regions of the Human Cortex, Corpus 
Callosum, Anterior Commissure, Amygdala and Hippocampus. 
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computers were not designed to be models of the brain even when they are 
running neural networks to simulate its behavior within their own compu- 
tational restrictions. Most fundamental questions are however related to 
its computational power, in both senses: time/space complexity and degree 
of solvability. Most computational brain models programmed to date are 
in fact strictly speaking, less powerful than a UTM. Researchers such as 
Stannett 63 have speculated that "if biological systems really do implement 
analog or quantum computation, or perhaps some mixture of the two, it is 
highly likely that they are provably more powerful computationally, than 
Turing machines. This statement implies that true human intelligence can- 
not be implemented or supported by Turing machines, an opinion shared by 
Roger Penrose 47 , who believes mechanical intelligence is impossible since 
purely physical processes are non-computable. A position strongly criti- 
cized by many researchers in the field since its authors first propagated the 
idea. 

However, assuming some kind of relation between the mind and the 
brain's physical actions, neural networks may be accepted as a model of the 
human mind operation. Since such a mind/brain relation is widely accepted 
in different ways and levels, we concern ourselves with the computational 
power of these devices and the features that such networks must possess. 

We will address which from our point of view, goes to the crux of the 
matter when the question of the computational power of the brain is raised, 
that is, its solvability degree. This means that we do not concern ourselves 
with what could be the recipe in which a simulation could run, since if we 
restrict ourselves to the discussion of artificial neural networks running on 
actual digital computers, we will be restricted to the lowest computational 
degree of solvability. From this, it can be easily deduced that if there is 
a fundamental difference between the architecture of the brain and digital 
computers, then the efforts of the artificial neural networks to fully simulate 
the brain either for the purpose of study or reproduction, are destined to 
have fundamentally different degrees of power. 

Based on certain references 5 as well as our own research, we have iden- 
tified at least five mathematical descriptions in which "super- Turing" ca- 
pabilities have been formally captured: super-tasks 4 ' 68 and accelerated 
Turing machines, Weyl machines or Zeus machines , Trial and Error 
machines 50 , Non-Standard Quantum Computation, and the Analog Re- 
current Neural Networks 59 . We have also identified other proposals con- 
cerning Turing machines with some kind of interaction between them or the 
environment 71,72,11 , those models provide a basis for the following claims: 
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(1) Minds are not computers, because (most) thought processes are not. 

(a) It can "violate" Godel's theorem, therefore is not a com- 
puting machine, a claim made most famously by Lucas 29 in 1961. 

(b) Mind can "solve" the "Entscheidungsproblem" , there- 
fore is not a computing machine. 

(2) Minds are computing devices but not of the same power of Turing 
machines (maybe Godel himself 1 ). 

(a) There are special operations that occur in the brain 
which are not Turing computable, a claim made most famously by 
Penrose 47 . 

(b) The mind could be a machine but with access to a cer- 
tain oracle (from an external source or from a previously coded 
internal source). 

All hyper-computational models presented to date are purely theoret- 
ical, and we may well ask whether they can actually be implemented in 
the sense that the universal Turing machine is implemented or pseudo- 
implemented in a common digital computer. A personal hyper-computer 
would be no more an implementation of a hyper-computer model, than a 
personal computer is of a UTM and nobody has physically implemented 
any kind of hyper-computer. Most models seems to take advantage of (a) 
a type of oracle device or (b) an infinite number of steps in a finite amount 
of time. 

Jack Copeland has pointed out 13 an interesting fact concerning the way 
Turing machines work. He stated that Turing machines are closed systems 
that do not accept input while operating, whereas the brain continually 
receives input from the environment. Based on this observation, Copeland 
has proposed the coupled Turing machine which is connected to the envi- 
ronment via one or more input channels 13 . However, as Christof Teuscher 
and Moshe Sipper have pointed out 65 , any coupled machine with a finite 



'in his 1951 Gibbs lecture 22 Godel attempts to use incompleteness to reason about 
human intelligence. Godel uses the incompleteness theorem to arrive at the following 
disjunction: (a) Either mathematics is incomplctcable in this sense, that its evident ax- 
ioms can never be comprised in a finite rule, that is to say, the human mind (even within 
the realm of pure mathematics) infinitely surpasses the powers of any finite machine, or 
(b) or else there exist absolutely unsolvable diophantinc problems (or absolute undecid- 
able propositions) for which it cannot decide whether solutions exist. Godel finds (b) 
not plausible, and thus he seems have believed that the human mind was not equivalent 
to a finite machine, i.e., its power exceeded that of any finite machine, the term used 
originally for Turing machines 
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input stream can be simulated by a UTM since the data can be written on 
the machine's tape before it begins operation. From dynamic systems we 
often decide almost in an arbitrary way when a system will be closed in 
order to handle it. However, the chain of such external systems, potentially 
infinite (even just by loops) can create a non-linear system which could 
truly be more complex and maybe more powerful. Some other claims and 
critics have been made in this regard. 

Some other authors claim that the "super-mentalistic" perspective is 
not a scientific one, as it implies the possibility of assigning non-reducible 
phenomena to some sort of information processing. However, we believe 
that this fact does not preclude a study on what formal properties can be 
required from non- Turing models of the human mind. A clarification of 
these properties would help us understand to what extent "super- Turing" 
models of the mind can or cannot be considered in a scientific way 

3. A computational model for the human mind 

3.1. Determining the power of a neural network by the 
complexity of its weights 

Neural Networks are able to encode information by several ways. If brain's 
neural network computes equal or less than the Turing degree of Turing 
machines without oracles, their weights should code only whole or ratio- 
nal numbers. However, if brain's NN is more powerful it is clear that it is 
possible to use an oracle to build a neural network that simulates a Turing 
machine with that oracle in order to recognize any arbitrary language 59 . 
This can be done by encoding the oracle into a real number r which the 
neural network "consults" in order to know if a certain input (or a trans- 
formation of it) does belong to the language encoded by r. 

Because we are interested in the set of Turing degrees of the weights of 
a neural network, and since not always Turing degrees are comparable, it 
is necessarily to use the notion of a maximal element. 

Definition 4- Let be a partially ordered set (A,<). Then an element 
r^A is said to be maximal if, for all a£A,r^a. 

We are going to take as "information" to the repository of languages. 
In Computer Science and Linguistics a language is a set of symbols from 



s Notc that the definition for a maximal element is true for any two elements of a partially 
ordered set that are comparable. However, it may be the case that two elements of a 
given partial ordering are not comparable as in the case of Turing degrees as Post proved 
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an alphabet. As it is well known, every string on an arbitrary alphabet 
can be encoded into a string on the binary alphabet {0,1}. In the same 
way, a real number with infinite expansion can also be represented by an 
infinite binary string. Putting together these facts, a language can be easily 
encoded into a real number r in the interval [0, 1) taking the n digit of the 
expansion of r as the value of the characteristic function of the language 
applied to the n string in {0, 1}* ordered lexicographically. Additionally, 
this encoding is unique. Then given a language L, is the real number 
that encodes it. 

Definition 5. A real number tl is computable if and only if it is the 
limit of an effectively converging computable sequence of rational numbers. 
The Turing degree of a real number tl is defined as the Turing degree of its 
binary expansion 11 . 

Example 1. This definition is quite natural and robust. For instance, 
let P be the set of natural numbers defined by P={neN|thc digit n of the 
binary expansion of (jr-3) is 1}. P is evidently computable. Let be RN a 
neural net with weights ir and n-n. Since ir is Turing-computable and ir-n 
too then RN will compute only those languages coded by n and ir-n. 

Example 2. However if h is used as a weight, where h is the Chaitin 
constant fl defined as the probability of halting of a Turing machine for 
an input Xi RN is going to compute definable languages but non- Turing 
computable. It is evident that RN computes h and those in its same 
complexity degree. If the set of maximals of RN has only h then the 
Turing degree of RN is going to be the Turing degree of h, that is 0'. Then 
if the human mind is capable to hold ir, h or any other value as a connection 
between neurons it will determine its computational power by taking the 
set of maximals of the weight's Turing degrees. 

3.2. Extracting information from the weights 

A more interesting question is how to verify what type of languages a 
weight has coded in order to determine the computational power of a neu- 
ral network. That means what language a real number encodes. Since the 
function of the encoding is bijective it is just needed to analyze one by 
one the digits of the binary expansion of the real number in order to re- 
construct the encoded language. Of course, languages encoded in this way 



h it can be seen as a subset of natural numbers (or in fact the whole set of natural numbers 
encoded in a single real number concatcning all). It is evident that not all real numbers 
are computable (they arc also identified as random in Chaitin theory). 
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go well beyond recursively enumerable languages and most of them will go 
beyond any Turing computable procedure. It is clear that if a weight really 
encodes a non rational number it is going to be impossible to extract all 
digits in a finite time and space for most of them. However, if all weights 
in the brain neural network are finite it will represent an argument for a 
computationalism position on the human mind (and depending upon the 
accepted relation between the brain and mind). If a computable irrational 
number is taken, for example, n, the language it encodes is clearly recur- 
sively enumerable but infinite. However this procedure obscures the rules 
(of a suitable Turing machine) that generate the language and it is very 
difficult to see how such rules could be derived from the number itself and 
then impossible to determine if the substring n(7r) for the first n digits of 
7T is really going to be it at the end or any other number sharing the first 
n digits. Even further, it seems to be impossible to distinguish between a 
computable or non-computable number by this way. 

Relativization of arbitrary neural networks by using the computational 
power of their weights by any procedure could allow us to classify them 
into the well known traditional hierarchies. For this purpose, we will use 
an oracle to build a neural network that simulates a Turing machine with 
that oracle to recognize a language. Let M be the neural network we will 
build. First, we encode the input w into a real number r, as we saw in the 
last section, then we code the oracle given in the same way. Let o be the 
real number that encodes the oracle. For each entry i coded in r, the oracle 
coded in o is consulted, when the answer is yes, the digit i is 1, when it is 
not, the answer is 0. 

Then M has two parts, one a sub-network N will simulate a Turing 
machine, and the other, named O will simule the oracle machine 59 . Let 
v(i) be the oracle answer for the input i, we denote by c the concatena- 
tion c = v(0)v(l)...v(i)...v(n), and by d(c) a Cantor encoding 59 of r. The 
Cantor encoding can be avoided since we are not concerned in this paper 
in time/space complexity. According to Davis 18 it is enough a straight 
encoding based on length and lexicographical order 1 

Then, the network M can be described 59 as a composition of two sub- 
networks: the first, N, for which r w is a weight, receives the input w, and 



'e— >1, a— >2, b^3, aa— >4, ab— >5, ba— >6, bb— >7, aaa^8, aab^9, aba^lO, . . . For example 
if L is defined by all the strings that begins with an "a" followed by an arbitrary number 
of "b"sL={a, ab, abb, abbb, . . .}, then the set S will be {2, 5, 11, 23, . . .} and the 
real number encoding L will be L r =0.0100100000100000000000100. . . (in fact it is a non 
rational number in this case). 
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after a fixed computation N submit y w to O, where y comes from w after 
the computation of the Turing machine simulation, the sub-network which 
performs the oracle machine and has d(c) — o as weight. The output of the 
sub- network O will depend on a binary activation function^ . 

The input arrives on two binary input lines. The first is a data line, 
which is used to carry a binary input signal, when no signal is present, the 
output is zero. The second is a validation line, which indicates when the 
data line is active. It takes the value one while the input is present and 
zero when not. There are also two output lines, that take the same roles. 
These conventions allow using all external signals to be binary. This gives 
rise to a taxonomy of computational capacities summarized in the table of 
the following section. 

If any arbitrary neural network TV is equivalent to a network N' with 
two sub-networks M and O that simulate the Turing machine and the 
oracle component respectively, then the computational power of the whole 
network N will be determined by the sub-network O. Therefore, it makes 
sense to classify a neural network by means of the Turing degree of its oracle 
component. The Turing degree of an oracle in a neural network is the set of 
maximals of the Turing degrees of the weights of the network. The Turing 
degree of each weight depends on its encoding capacity. 

3.3. Other sources of computational power: spikes and 
operations 

In recent years, data from neurobiological experiments has made it increas- 
ingly clear that biological neural networks, which communicate through 
pulses, use the timing of these pulses to transmit information and to per- 
form computation. This realization has stimulated a significant growth of 
research activity in the area of Spiking (or Pulsed) neural networks (SNNs 
or PNNs), theoretical analysis, as well as the development of new ANN 
paradigms 26 ' 35 . From the perspective of our main concern in this paper, 
an important question is: What type of information in terms of complex- 
ity, might be carried by an action potential? Each action potential could 
represent a single bit of information, similar to a serial digital communica- 
tion channel without error checking. However, the information per action 
potential may differ according to the function of the neural network. In 



J A sigmoidal type function called the signal function defined as: signal(x) = if x<0 
and 1 in other case. 
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traditional neural network models the timing of computation steps is usu- 
ally "trivialized" . Clearly, precisely timing of spikes would allow neurons to 
communicate much more information than with essentially random spikes. 
Otherwise, such temporally-coded information is lost as a source of possi- 
ble computational power in those traditional models. It has been shown 
that the use of pulse correlations in addition to pulse rates can significantly 
increase the computational power of a neural network. These timing en- 
codings finally contribute to the overall power of such networks. 

Biologically-inspired computation paradigms take different levels of ab- 
straction when modeling neural dynamics. The production of action po- 
tentials or spikes has been ab- stracted away in many rate-based neuro- 
dynamic models, but recently this feature has gained renewed interest. In 
biological neural networks, information is transmitted by the conduction 
of action potentials along axons, and information processing takes place at 
the synapses, dendrites, and soma of neurons 28 . It has been shown that 
firing correlations play a significant computational role in many biological 
neural systems. 

Models of biological neural networks in terms of dynamic systems have 
been studied and formalized by Hodgkin-Huxley 23 and FitzHugh-Nagumo 
equations 20,40 and others. Recently experimental evidence has accumu- 
lated during the last few years, which indicates that many biological neural 
systems use the timing of single action potentials (or "spikes") to encode 
information 1 ' 3,6 ' 51 ' 24 ' 21 ' 70 ' 53,55 ' 56 ' 66 . Experiments have shown that in vitro 
biological neurons fire with slightly varying delays in response to repetitions 
of the same current injection 2 . Even when this behavior may be explained 
by noisy versions of neural networks, the source for some kind of possible 
codification in such processes remains. However, noise certainly affects the 
computational power of networks of spiking neurons for analog input 31 ' 39 . 
Nevertheless, empirical experiments in which complex fluctuations exist, as 
seen for example in EEG k signals that are generally taken as noise, may be 
indicative of complex dynamics for processing information similar to that 
of pulse propagation networks or spiking neurons, maybe continuously over 
time. 

A central theme in this paper is the coding capabilities of neural net- 



k Electroencephalography is the neurophysiologic measurement of the electrical activity 
of the brain by recording from electrodes placed on the scalp or, in special cases, on the 
cortex. The resulting traces are known as an electroencephalogram (EEG) and represent 
so-called brain-waves. 
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works in which its computational power lies. Most accepted descriptions of 
the human neural network involve finite and discrete quantities, but some 
of them remain as continuous variables from one processing. Computer 
scientists have been studying ANNs for many years. Although ANNs were 
inspired by real biological networks, typical ANNs do not model a number 
of aspects of biology that may prove to be important. Real neurons, as we 
have seen for example, communicate by sending out little spikes of voltage 
called action potentials (APs). ANNs, however, do not model the timing of 
these individual APs. Instead, ANNs typically assume that APs are repet- 
itive, and they model only the rate of that repetition. However some of the 
computational power of a biological neural network could be derived from 
the precise timing of the individual APs or other properties inherent to the 
biological neural network that it is not fully right-modeled. Regular ANNs 
could never model such a possibility defined as their currently are. 

Additionally, all ANNs running over digital machines are incapable of 
simulating any possible analogical signal or full irrational encodings if they 
were present in human brain's architecture. Methods for estimating the 
computational power of neural circuits and their relation to artificial neu- 
ral networks models have been established. Maass 30 ' 32 ' 33 ' 34 and Markram 38 
have recently argued that "in contrast to Turing machines, generic compu- 
tations by neural circuits are not digital, and are not carried out on static 
inputs, but rather on functions of time". These kinds of experiments could 
ultimately provide a definitive answer to the critical issues which are the 
concern of this paper. 

These models are also interesting because even if a neural network is 
constrained to simple weights, namely whole or rational numbers, it is pos- 
sible to achieve an extra power from the timing process, encoding what 
is not possible to encode in the weights. Since they are similar in power 
to analog signal processing, they can be compared to the traditional hier- 
archies we have already explored. Even when a mathematically rigorous 
analysis of the computational power of networks of spiking neurons has so 
far been missing, they are equivalent to the levels of such hierarchies, sim- 
ply by replacing weights encodings with pulse timing encodings. In other 
words, the computational power is transferred from weights to spikes, and 
in the presence of a mixed model with both weights and spikes encod- 
ings, its final computational power will be the power of the most powerful. 
Both pulse frequencies and correlations which are computationally relevant 
can be seen as operations involving potentially any possible real numbers 
in principle equivalent to a weight encoding. Hence, the overall power of 
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these PNNs or SNNs is determined by the maximal Turing degree of the 
union of the Turing degrees of both weights and time encodings. It has also 
been shown that SNNs are, with regard to the number of neurons that are 
needed, computationally more powerful than other neural network models. 
These models are obviously Turing reducible when variables (the input and 
all internal operations) are restricted to Turing-computable values if such 
model is close under its operations. However when arbitrary values are 
allowed then they can compute possible non- Turing functions. Maass 36 has 
shown that, with regard to temporal coding of real- valued variables any 
continuous function can be approximated arbitrarily closely. In this case, 
the model depends on the continuity of time. 

On the other hand, since it is possible to compute y/2 from simple opera- 
tions between whole numbers (as the hypotenuse of a Pythagorean triangle 
with unitary sides) and even to make proofs of irrationality without calcu- 
lating their decimal expansion, it might be possible to prove if brain's neu- 
ral network can hold non-computable numbers if it is able to perform some 
special operators where the discrete operations, such as primitive recursion 
or bounded sums and products, are replaced by operations on continuous 
functions, such as integration or linear integration 43 . The sets of those 
real functions 54 can be definable by a general-purpose analog computer 57 
or GPAC, which is a general computing model evolving in continuos time 1 . 
Then even if weights and spikes are restricted to Turing-computable func- 
tions, the brain neural network might be not closed if at least an operation 
is not recursive possibly (see 15 and 16 ) taking the involved values to non- 
Turing degree at some point of the computation. Traditional functional 
operations like Sequential Composition, Parallel Composition or definition 
by Cases, Primitive Recursion or simply Recursion, Bounded Recursion, 
Partial Projection, Cut-off Subtraction and the order functions like the 
Minimization or Projection and Bounded minimization take computable 
functions into computable functions. By contrast, It is well known that 
limits take computable functions beyond the class defined by the power of 
Turing machines. For a more detailed description of such operations see 
8 . The power of human brain operations remains unknown, but even if 



162,16 ip^g c j ass f /^-recursive functions is very large. It contains many traditionally 
non-computable functions, such as the characteristic functions of sets of the arithmeti- 
cal hierarchy 42 ' 45 . Experimental proofs beyond this level could be more difficult if not 
impossible since the construction of a sequence of real numbers which can not be com- 
putably diagonalized is used to prove that there are continuous functions without a 
Turing degree. 
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non-traditional operations were possible both theoretical and empirical ev- 
idence seem to be hard or impossible since the verification of non- Turing 
operators require non- Turing inputs and outputs which seems to be undis- 
tinguishable from those Turing-computable. Nevertheless neurological tests 
could be designed in order to achieve some advance in this direction. 

One could suppose that given the evolution of neural networks models, 
several better approaches to the biological neural network model could be 
expected. Other possible sources could be unrepresented in current artificial 
models. Experimental and theoretical research in the field should continue 
to relate it more to the biological model. Maass 36 suggests SNNs as the 
third generation of the artificial neural network model. Each one seems 
to offer better approaches to the experimental evidence of the brain's own 
neural network. 

Now, we can build a simplified hierarchy of computational power re- 
garding the two possible sources explored in this paper (weights and spikes, 
which are equivalent in terms that they can be replaced one for the other 
preserving the whole complexity of the NN): 



Neural Network Architec- 
ture 


Computational Power 


R Non-computable num- 
bers 


Turing machine with an oracle depend- 
ing on the maximal Turing degree of of 
the weights: w\,W2, ■ ■ ■, n <G R and pulses 
Pi,P2, ■ ■ ■, m € R, not always in the Arith- 
metical Hierarchy 


Q Rational numbers 


at most Turing machine 


Z Whole numbers 


at most bounded automata 



At the bottom level, we have neural networks computing Turing- 
computable languages if the oracle is empty. On the other hand, weight's 
and spikes' complexity could be determined -assuming that the neural net- 
work is closed under isomorphism- then they can be decoded into an oracle 
which will determine its computational power. If the neural network per- 
form not closed operations under isomorphism, the neural network will 
compute in a range determined by the bounds of such operations, possibly 
determining an interval within this simplified hierarchy. 
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4. Discussion and Conclusion 

Even when the answer concerning the computational power of the human 
mind should come from neurophysiological and interdisciplinary research, 
including the correspondence between the natural and the artificial neural 
network models, the question remains of mathematical and philosophical 
interest as it is possible to explore all possible scenarios inside the com- 
putability theory. We add that if we were able to determine some key 
properties of the physical universe and the mind we could determine the 
computational power of the mind, among them: 

• if our physical universe is discrete or continuous, and in which case 

• if it can hold non-computable numbers; 

• if it can hold non-computable operations; 

• if the brain inherits those properties; 

• if the mind can take advantage of them, which depends on the 
precise relationship between the mind and the physical operation of 
the brain. 

As it can be seen, these are not small requirements. However once we 
can ascertain some fundamental properties of the mind it could be possible 
to know to which computational level it belongs by its maximal Turing 
degree. The value of this paper lies in its effort to restore some of the 
claims which have been made and proven for artificial neural networks to the 
natural model and the problem of determining the mind's computational 
power. Then, it is not just about the way in which a neural network can 
achieve one or more computational powers, but also which features a brain 
should have to potentially take its power to one level or another: either 
less, equal or more powerful than Turing machines. 

Then, if every language with a given alphabet can be encoded and po- 
tentially extracted into a real number (from weights or spikes), this real 
number can be used as the main component of a neural network that simu- 
lates an oracle Turing machine whose oracle is the characteristic function of 
the language. Therefore by determining the neural network encoding capa- 
bility and knowing the type of operations permitted in it, its computational 
power could be determined. 

We would like to conclude, by remarking that a full model of the mind 
has a very difficult task ahead. We hope that this paper has succeeded in 
shedding light on the current research and ways to build certain bridges 
from the artificial to the natural model, as a possible way to eventually 
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determine the computational power of the human mind taking in consider- 
ation all possible sources of power. 
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